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Abstract 

This study investigates students’ achievement goal orientation in relation to their attitudes toward group 
work and perceptions of group members’ collaboration behaviors. A total of 48 (33 males and 15 females) 4 lh 
year Instructional Technology Department students were taking Design, Development and Evaluation of 
Educational Software course at Middle East Technical University in the Spring term of the year 2003 formed the 
sample of this study. The study lasted 14 weeks. During this period the students worked in groups and 
developed 3D learning environments by using rapid prototyping approach and Active World. 3.3®. At the end of 
the semester a four-part survey questionnaire was given to students to collect data on background, achievement 
goal orientations, attitudes toward group work and perceptions of their group members’ performances in terms of 
effective collaborative rubric. The data were analyzed by descriptive and inferential statistics. The correlation 
analysis showed no significant relationship between attitudes toward group work and profiles of achievement 
goal orientations. However there is a correlation between attitudes toward group work and how subjects perceive 
their team members’ collaboration in regard to contribution to task, discussing and listening, and team 
functioning behaviors. Significant correlations were found between collaboration on contribution to task, 
discussing and listening, and team functioning behaviors. The results indicated that there was a significant 
relationship between mastery and ego orientations, negative relationship between GPA and work avoidant 
orientation. Finally males were significantly more work-avoidant than females. 



Introduction 

In the process of designing and developing instructional software, certain foundations 
are considered. Alessi and Trollip (2001) defined these foundations as standards, ongoing 
evaluation and project management. The third one, project management, should penetrate the 
entire project in regard to good management of the tasks, resources, money, and time. From 
the starting point, it is important that the project is under tight control in regard to the stated 
issues. To make it possible, instructional software development project requires “a team of 
talented individuals to work toward common goal” (Alessi and Trollip, 2001, p.530). Team 
members should communicate and collaborate well for the success of the project. 

Instructional software design and development subject can be learned better through a 
project-based collaborative learning experience since in real settings this process requires a 
strict teamwork. According to Shanley, (1999), a technology coordinator Lynne Sueoke views 
project-based learning as a tool that “empowers children to understand exactly what is they are 
learning” (p.2). It is a challenging process but when thi nk about the gains and outcomes, it is 
worth to effort. They stated that “project-based learning is much more relevant, and helps to 
build valuable critical thinking and problem-solving skills” (p.2). However, individual 



characteristics of team members such as achievement goal orientation, attitudes toward group 
work and collaboration affect project-based learning process and the outcomes of the process. 

In reaching instructional goals, students’ perceptions of achievement, understanding of 
learning, studying habits, and interactions with others in the teaching and learning 
environment are some of the determining factors. As Eggen and Kauchak (1999) stated, one 
way of reaching these goals is “guiding students in setting their own goals” (p.417). While 
guiding students, the distinction between performance oriented and learning oriented goals are 
important. 

Eggen and Kauchak (1999) defined performance goal as “focus on demonstrating high 
ability and avoiding failure. In a perfonnance orientation, learning isn’t viewed as a goal in 
itself, but rather as a means to end, such as a high-test score or good grade” (p. 418). Learning 
goals, on the other side, focus on the challenge and mastery of a task (Pintrich & Garcia, 1991; 
Stipek, 1996, cited in Eggen and Kauchak 1999). “Learning goals lead to task orientation, in 
which students focus on understanding and don’t worry about failure or comparisons with 
others” (Eggen and Kauchak 1999, p. 418). 

To be able succeed in project-based learning, collaboration among the project team 
members should be emphasized. According to Alessi and Trollip (2001) collaborative 
learning is “... suggesting environment in which learners work on a shared project or 
goal”(p.34). Grabe and Grabe (2001) indicated that “collaboration have been purposefully 
structured according to specific and clearly identified principles” (p. 70). In this process they 
work together to achieve the goals or to finish the project, they learn from each other, they 
express their own ideas and understanding to help others understand them, they develop 
understanding of other perspectives and views as the main advantages of collaborative 
learning. “The group process naturally produces a level of cognitive conflict that challenges 
the personal understanding of group members and encourages more active self regulated 
learning” (Grabe and Grabe, 2001, p.71). However collaborative learning activities are 
difficult to organize. As Alessi and Trollip (2001) and Grabe and Grabe (2001) stated, the 
main disadvantage of collaborative learning is that some learners may benefit more than the 
others in such environment. To have all learners benefit from this experience in an optimum 
way, activities should be planned, and type of grouping from one subject matter to another 
should be defined well. 

Learning instructional software design and development subject requires a project-based 
collaborative learning setting in which students learn from each other, reflect their own and 
team members’ ideas, to experience group interdependence which as a main ingredient in this 
process. However, learners’ perceptions of achievement goal orientation, and how they see 
group work may affect the success of this process. 

Although there are many research studies that investigate goal orientation, collaboration, 
attitudes toward group work, and instructional software development process separately in the 
literature, there are not many studies that examine the relations among these themes. 
Therefore, this study tries to find out if there is a correlation between goal orientation profiles, 
group work attitudes and group members’ perceived collaboration behaviors in instructional 




software design and development process, and difference between male and female students. 
The specific research questions that guide this study are the following: 

(1) What are the participants’ achievement goal orientations, attitudes toward group work 
and perception of their group members’ performances in terms of effective 
collaborative behaviors? 

(2) Is there a significant difference between male and female students’ achievement goal 
orientations, attitudes toward group work and effective collaboration behaviors? 

(3) Is there a significant relationship between students’ achievement goal orientations, 
attitudes toward group work, effective collaboration behaviors and GPAs? 



Method 

A case study design was used to examine the participants’ achievement goal orientations, 
attitudes toward group work, effective collaboration behaviors, and the relationships among 
these variables in a project-based learning process. The study included fourth-year Computer 
Education and Instructional Technology students at Middle East Technical University in 
Ankara, Turkey. For this purpose a specific course “Educational Software Design, Development 
and Evaluation” was selected, and the students taking the course formed the sample of the 
study. A survey technique was used to collect the related data. Below, the detailed description 
of the study, subjects, development of data collection instruments, procedures are presented. 

Subjects of The Study 

In order to investigate students’ achievement goal orientation, attitudes toward group 
work and perceptions of group member’ collaboration behaviors, a specific undergraduate 
course “Design, Development and Evaluation of Educational Software Course” (at Computer 
Education and Instructional Technology Department at Middle East Technical University in 
Turkey) which was taken by the 4 th year Instructional Technology students was selected. The 
course, involving three theoretical hours and 2 practice hours was offered in the Spring term 
of 2003 in two sections by two instructors. In both sections the same content was covered and 
the same instructional materials and methods were used. The 4 th year students taking that 
course formed the sample of this study. A total of 56 students were enrolled in the course 
(sections one 27, and two 29), but a total of 48 (33 males and 15 females) subjects volunteered 
to participate in the study. 14 project groups were formed based on gender and Cumulative 
Grade Points Average. Students were assigned to the project groups randomly considering 
gender and achievement (high, average and low achievers) clusters to make sure that students 
from both genders and different achievement levels were represented in the project teams. 
Design, Development and Evaluation of Educational Software course was regarded as their 
graduation course, and they were expected to implement what they had learned during 
previous undergraduate years to their projects in this course. After graduation most students 
either work as computer teachers at primary education level or work in private sector on 
educational software development. 



Procedures of the Study 



The study lasted 14 weeks. During this period the group members worked together and 
developed instructional software by using rapid prototyping approach. In the beginning of the 
semester the students were infonned that as the course project they were supposed develop a 
three-dimensional (3D) game like learning environment by following rapid prototyping 
approach. The course consisted of two parts. The first part (six weeks) involved providing 
bases for students in the educational software development field, the second part (remaining 
eight weeks) focused on software development process. 

In the first six weeks of the course the students were taught theoretical bases of 
educational software development process on “Learning Principles and Approaches, General 
Features of Educational Software, Games and Simulations” (Alessi and Trollip, 2001), 
ADDIE and Rapid Prototyping Models. During this period an expert from a private 
educational software development company came and explained how educational software 
development procedures took place in real life settings. 

During the practice hours of the first part of the course, the students played with three 
educational games and wrote a reflective journal individually to compare them in terms of 
their educational aspects. For the remaining time they examined the characteristics of the 3D 
environment development software (Active World 3.3 ) which they did not know then and 
used in their projects. At the same time they as group members brainstormed on their projects 
ideas,, which needed to be in line with the course framework. 

In the second part of the course, the students focused on their projects. They had to 
perform analysis, design, development and implementation/evaluation phases of educational 
software development process, and had to write reports related with each phase. During this 
period each group was assigned a specific weekly time period for consultation, and the group 
members were consulting their instructors on their projects and reports. At the end of the each 
consultation period, each group received feedback related with their projects (paper-based 
prototype, computer-based prototype, the actual product and so on) and reports. In regard to 
feedback, they modified their prototypes. This procedure was an iterative procedure, and until 
the end of the project there were revisions. At the end of the semester the groups together 
with the documentation parts finished their 3D game like learning environments. After that to 
share what they had done, they presented their projects to their classmates. 

Instruments 

A four-part survey questionnaire was developed to collect data on background variables, 
students’ achievement goal orientation, students’ attitudes toward group work and their 
perceptions of their group members’ performances in tenns of effective collaborative rubric. 

The Achievement Goal Orientation Questionnaire included statements related to the 
three orientation subscales, mastery (indicates a learning goal and consists of 13 items), ego- 
social, and work avoidant (indicate performance goal and consists of 13 and 8 items 
respectively). A five point Likert-type scale was used to differentiate orientations from 1 as 
low and 5 as high orientation, and Achievement Goal Orientation Questionnaire was 
developed originally by Somuncuoglu and Yildirim (1998) and was adapted for this study. 




They carried out a pilot study with a group of 47 students in 1995 and found a .85 alpha score 
on mastery, a .83 alpha score on ego-social, and a .79 alpha score on work-avoidant scales. 

The Effective Collaborative Rubric Survey consisted of three sections: contribution of 
group’s tasks and completion of personal tasks (5 items), discussion skills and active listening 
(4 items), and contribution to group’s evaluation, problem solving and team functioning (3 
items). It was a four-point scale instrument ( 1 being the lowest level collaboration, 4 being the 
highest level collaboration). 

Attitudes toward group work questionnaire consisted of 12 Likert-type items where 1 
indicating the lowest level and 5 indicating the highest level agreements with the statements. 

Both attitudes toward group work questionnaire and the effective collaborative rubric 
questionnaire were adapted from Information Society for Technology in Education Web site. 
For this study, a .80 alpha score for attitudes toward group work questionnaire, and a .93 alpha 
score for effective collaboration rubric were found. 

The data gathered with the questionnaire were analyzed by descriptive and inferential 
statistics. First descriptive analysis of the items in three scales in terms of means and 
percentages were carried out. Second, sub-scales scores were calculated for each category 
under achievement goal orientation scale and collaboration rubric scales then relationship 
between achievement goal orientations, attitudes toward group work and collaboration rubrics. 
After, the relationship between achievement goal orientations, attitudes toward group work, 
and the effective collaboration rubric scores were analyzed through a correlation coefficient 
test. Finally t-test was perfonned to find out if there is a significant difference based on 
gender. 

Rapid Prototyping Model 

In game like learning environment development process, the project groups used rapid 
prototyping approach that was adapted to educational software development field from 
software engineering field by Tripp and Bichelmeyer. (Tripp and Bichehneyer, 1990). 
According to Lantz (cited in Trip and Bichehneyer, 1990) prototyping means" system 
development methodology based on building and using a model of a system for designing, 
implementing, testing and installing the system" (p. 35). In this approach, after describing the 
needs and objectives briefly, research and development processes were conducted parallel to 
each other to create prototypes of the software. After testing, the developer may come up with 
the final product or not (Tripp and Bichelmeyer , 1990) 

During the progress of their group projects, the students wrote analysis, design, 
development, and implementation / evaluation reports inline with rapid prototyping approach 
and received weekly feedback from the course instructor. Parallel to the reports they 
developed first paper-based, second computer-based and third Active World based prototypes. 
They gathered feedbacks for their prototypes from the instructors, from their classmates, and 
from several related people (teachers, or students) and came up with the final product. 




Active World. 3.3 



The students in this study used Active World 3.3 to develop a 3D game like learning 
environment. The access to the software was provided over Indiana University in Indiana, 
United States. Active World interface consists of six main components: 3D virtual world, 
Web, chat, tabs, toolbar and menu bar. 3D virtual world part allows users to travel in the 
virtual space; Web component provides the knowledge base for the subject area and browsing; 
chat component helps users converse with others in the system simultaneously; tab parts 
provides moving from one virtual world to the another one. Menu and tool bars help users 
build virtual worlds and change preferences. The students in the study were able to use Active 
World at the departments’ computer laboratories and needed to connect Indiana University’s 
server trough Internet. 



Limitations of the Study 

Small sample size is one limitation of the study. The number of the participants in the 
study was limited to the number of fourth year students in Instructional Technology Department 
and to the students who were volunteers for this study. Therefore the results of the study cannot 
be generalized directly beyond the case study group. Another limitation arises from the type of 
and way of software used in the study. As indicated Active World. 3.3 was used in this study. 
The space of the virtual world provided for each project group by Indiana University was 
limited, and connecting to Indiana University servers through the Internet was too slow. In 
addition the students were using 3D development software for the first time. Even though the 
students overcome the novelty of the development tool, limited virtual space and slow Internet 
access remained as the main limitations of the project groups. Despite these limitations, this 
study provides valuable contribution with regard to relationship between goal orientation 
profiles, group work attitudes and collaboration behaviors in instructional software design and 
development process. 



Results 



Descriptive Results on Achievement Goal Orientations, Attitudes Toward Group Work 
and Effective Collaboration Behaviors 

Descriptive results of the study are presented in Table 1 below. The results of the study 
showed that subjects are very close to mastery goal orientation (M=3.87), then ego-social 
(M=2.97), and the last work avoidant (M=2. 1) as a whole. The dominant orientation profiles 
among the students are mastery, and both mastery and ego-social orientations. The findings 
indicate that majority of the students focus on the challenge and mastery of the task as Eggen 
and Kauchak (1999) indicated. 

Attitudes toward group work scores (M=3.56) showed that majority of the students 
agreed with the statements. It can be concluded from this result that students have positive 
attitude toward group work. 




Students had similar scores on contribution of group’s tasks and completion of personal 
tasks (M=3.46); discussion skills and active listening (3.46); and contribution to group’s 
evaluation, problem solving and team functioning (3.43) sub-scales of collaboration rubric 
indicating that students perceived other group members were effective collaborators (4 th level) 
in this process. 



Table 1. Descriptive Results on Achievement Goal Orientations, Attitudes 
Toward Group Work and Effective Collaboration Behaviors 





N 


Minimum 


Maximum 


Mean 


Std. Deviation 


Mastery 


48 


2.75 


5.00 


3.87 


.51 


Ego 


48 


1.38 


4.15 


2.97 


.71 


Work Avoidant 


48 


1.00 


3.75 


2.10 


.80 


Group work attitude 


46 


2.17 


4.67 


3.56 


.62 


Task collaboration 


48 


2.40 


4.00 


3.46 


.43 


Discussion collaboration 


48 


2.33 


4.00 


3.46 


.43 


Team functioning 
collaboration 


48 


2.00 


4.00 


3.43 


.46 



Differences in Achievement Goal Orientations, Attitudes toward Group Work and Effective 
Collaboration Behaviors by Genders 

As it is presented in Table 2, t-test results showed no significant difference between 
males and females’ attitude toward group work, and perceptions of effective collaboration 
rubric. In relation to achievement goal orientation profiles, t-test results indicated similarly 
that there were no significant differences between males and females on mastery orientation 
and ego-social orientation. However there was a significant difference between males and 
females in work avoidant profile indicating that males (M=2.25) are significantly more work- 
avoidant than females (M=1.75). 



Table 2 Differences in Achievement Goal Orientations, Attitudes 
Toward Group Work and Effective Collaboration Behaviors by Gender 





GENDER 


N 


Mean 


Std.Deviation 


Group work attitude 


male 


33 


3.55 


.60 




female 


15 


3.57 


.67 


Task collaboration 


male 


31 


3.51 


.40 




female 


15 


3.33 


.47 


Discussion collaboration 


male 


31 


3.50 


.40 




female 


15 


3.35 


.48 


Team functioning collaboration 


male 


31 


3.47 


.43 




female 


15 


3.35 


.50 


Mastery 


male 


31 


3.82 


.51 




female 


15 


3.98 


.50 


Ego 


male 


31 


3.00 


.73 




female 


15 


2.90 


.66 


Work Avoidant * 


male 


33 


2.26 


.80 




female 


15 


1.75 


.71 



* t(46)= 2.092, p=.042 



Relationship between Achievement Goal Orientations, Attitudes toward Group Work, 
and Perceptions of Effective Collaboration Behaviors and GPAs 

Correlation results are presented in Table 3. Regarding the relationship between 
attitudes toward group work and achievement goal orientations, the correlation analysis 
showed no significant relationship. However, there is correlation between attitudes toward 
group work and how subjects perceive their teammates’ collaborations in regard to discussion 
and active listening behaviors. The results indicated significant negative correlation between 
work avoidant orientation and students’ GPAs (-403). In collaboration behaviors, there is a 
significant correlation between completion of/contribution to tasks collaboration behaviors 
and discussion/active listening behaviors (.883), between completions of/contribution to tasks 
collaboration behaviors and team functioning behaviors (.822), and between discussion/active 
listening behaviors and team functioning behaviors (.783). In addition, the results showed that 
there is a significant correlation between mastery goal orientation and ego goal orientation 
profiles of the students (.437). 



Table 3 Relationship between Achievement Goal Orientations, Attitudes toward Group Work, 



Effective Co 



laboration Behaviors and GPAs 







1 


2 


3 


4 


5 


6 


Work 

Avoidant 


GPA 


P. Correlation 


-.171 


-.232 


-.278 


-.061 


.266 


-.130 


-.403** 


Sig. (2-tailed) 


.255 


.113 


.056 


.682 


.068 


.380 


.004 


N 


46 


48 


48 


48 


48 


48 


48 


1. Group work 


P. Correlation 




.372 


.438** 


.334 


.058 


-.037 


.009 


attitude 


Sig. (2-tailed) 




.011 


.002 


.023 


.701 


.806 


.955 


N 




46 


46 


46 


46 


46 


46 


2. Task 
collaboration 


P. Correlation 






.883** 


.822** 


-.140 


-.149 


.329 


Sig. (2-tailed) 






.000 


.000 


.342 


.313 


.023 


N 






48 


48 


48 


48 


48 


3. Discussion 
collaboration 


P. Correlation 








.783** 


-.219 


-.101 


.305 


Sig. (2-tailed) 








.000 


.136 


.495 


.035 


N 








48 


48 


48 


48 


4. Team 

functioning 

collaboration 


P. Correlation 










.000 


-.169 


.154 


Sig. (2-tailed) 










1.000 


.250 


.295 


N 










48 


48 


48 


5. Mastery 


P. Correlation 












.437** 


-.230 


Sig. (2-tailed) 












.002 


.116 


N 












48 


48 


6. Ego 


P. Correlation 














.190 


Sig. (2-tailed) 














.196 


N 














48 



** Correlation is significant at the 0.01 level (2-tailed). 



Conclusion 

The results of the study showed that the majority of participants are close to mastery 
orientation indicating that rather than thinking about failure or comparing themselves with 
other students, they focus on learning the subject (Eggen and Kauchak, 1999). Since they 
were to be graduated, they wanted to be ready for the job market and they might want to 
develop themselves in the field. In addition, up to some extend the students are ego social. 
They might think that their achievement and high perfonnance should be rewarded and 
honored, and they probably see grading as reward for their achievement. At the same time, 
high grades will be reference for them to look for a good job in the market in Turkey. In 



addition to mastery of the tasks other important thing for them can be grade since it is 
important to pass and after graduation. 

Majority of the students agreed with the statements in the group work attitude survey 
showing that they had positive attitudes toward group work. This result was probably due to 
the fact that they were close to graduation and they realized that they were going to work with 
someone who they did not know. Another reason can be the procedures followed in other 
courses at the department. Majority of the courses in the department have at least one long- 
term group project. From the first year, they are used to project based learning. The positive 
attitude toward group work might be due to the previous experiences about the group work. 

The students in the project groups had known each other but they did not work together 
in the previous projects. Even they did not work together as group before, collaboration 
results showed that students in this study were effective collaborators according to their 
teammates. It can be said that in this project-based learning process as Grabe and Grabe 
(2001) indicated they worked together to achieve the group goals, they contributed to and 
completed tasks, actively participated to discussions, and listened to others, solved problems 
to function as a team. This might be due to their experiences in the department and the nature 
of the project that they cannot handle the project alone. 

In this study males are significantly more work avoidant than the females. We can 
conclude from this result that female students are more involved with the software 
development project. Correlation analysis indicated that there is negative relationship 
between GPA and work avoidant orientation showing that low achievers do not focus on 
mastery of or learning the subject and they avoid these issues. 

Attitude toward group work seems to be correlated to participation to discussion and 
listening to others collaboration behaviors. The more positive the attitude toward group work, 
the more involvement with group discussion. In addition, there are positive correlations 
between the three sub categories of collaboration behaviors. This result specifies that these 
collaboration behaviors are interrelated and to function efficiently as a group, the members 
should have these collaborative behaviors. Among achievement orientations, there is a 
significant correlation between mastery orientation and ego social orientation. As it is 
indicated above as a result of mastery students needed some kind of promotion or reward. 

They might want justification and recognition for their learning as grades and social 
approvals, and as reference for their achievements. 

Investigating the relationship between achievement goal orientations, attitudes toward 
group work and students’ perceptions of group members’ collaboration behaviors is important 
from several perspectives. First of all to be successful, educational software development 
process requires effective teamwork and project management. For that reason, each team 
member’s goal orientation, their attitudes toward group work and their contributions to 
collaboration process are important for the outcome of educational software development 
process. Since each individual has different goal orientation and attitude toward group work, 
it would be valuable to examine which type of goal orientation is effective in different stages 
of software development process. The results of this study offer some explanation in terms of 




influence of goal orientations on attitudes toward group work and collaboration in educational 
software development process. In addition, the findings of this study will provide valuable 
information for educational software development practitioners in forming the development 
team to benefit more from this process. Finally, to have students benefit more from “design, 
development and evaluation of educational software development course,” this study provides 
insights for the instructors of this course in forming the project groups. 
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